Outcome differences by sex in oncology clinical trials

Identifying sex differences in outcomes and toxicity between males and females in oncology clinical trials is important and has also been mandated by National Institutes of Health policies. Here we analyze the Trialtrove database, finding that, strikingly, only 472/89,221 oncology clinical trials (0.5%) had curated post-treatment sex comparisons. Among 288 trials with comparisons of survival, outcome, or response, 16% report males having statistically significant better survival outcome or response, while 42% reported significantly better survival outcome or response for females. The strongest differences are in trials of EGFR inhibitors in lung cancer and rituximab in non-Hodgkin’s lymphoma (both favoring females). Among 44 trials with side effect comparisons, more trials report significantly lesser side effects in males (N = 22) than in females (N = 13). Thus, while statistical comparisons between sexes in oncology trials are rarely reported, important differences in outcome and toxicity exist. These considerable outcome and toxicity differences highlight the need for reporting sex differences more thoroughly going forward.

Reviewer #2: Remarks to the Author: This study addresses an important set of questions--whether oncology trials report statistical comparisons between sexes and when they do whether they find important differences in outcomes or toxicity.More, its sample is uniquely large, analyzing 89,221 trials in trialtrove.However, the paper is a bit hard to follow, for a few reasons, which are addressable.First, it would be helpful to have a formal methods section between the introduction and results sections of the paper.Currently, some methods are briefly described in the introduction and the results section, but the actualy methods section is at the end of the paper.Further there seem to be some new results presented in the discussion section (ie the total number of patients enrolled in the analyzed trials).There are some fragments and incomplete sentences in need of revision, for example, see line 87, which states, "Recent large studies that similarly used Trialtrove include."In terms of the analyses, it could be helpful to add a sensitivity analysis comparing reporting of statistical comparisons between sexes in clinicaltrials.govand or publications with what is in trialtrove.As currently structured, the paper reads more about what is curated by trialtrove than more generally in clinicaltrials.govor the medical literature.If these data exist and trialtrove has not prioritized curating them -that could be an important finding.Without the analysis I suggested I am unsure how representative these findings are of real world practices versus trialtrove curation practices.Overall, an important topic.
Reviewer #3: Remarks to the Author: This is a retrospective analysis to explore the outcome differences by sex in oncology clinical trials.Generally, this study highlights an important issue that statistical comparisons between sexes in oncology trials are rarely reported, meaningful differences in outcome and toxicity exist.To some extent, this paper is well-written and has relatively high clinical implications.However, the manuscript can be improved and the following comments need to be adequately addressed: 1.For figure 2-4, the authors indicate the outcome differences of clinical trials through "male", "female", and "same".I cannot think the outcome differences of these trials indicated by the "same" reported the same outcome differences between male and females.In fact, it is that the differences are not statistically significant.So the authors should kindly revise this information.
2. Authors revealed that only 472/89,221 oncology clinical trials 25 (0.5%) had curated posttreatment sex comparisons.Whether this result indicate the sex difference analysis is not mandatory for clinical trial design?If yes, whether the conclusions of this study can be applied for further trials?3.In Methods, authors should the clarify the update time of ClinicalTrials.gov in Trialtrove database.Meanwhile, please provide detailed information when using Python.
4. Authors should provide more in-depth discussion of how this study can impact existing and further clinical trial policies.

Responses to reviewers:
Preliminary remark: We thank all three reviewers for excellent suggestions that guided us to improve our manuscript.Numerous new references suggested directly and indirectly by the reviewers appear after our responses to the comments from reviewer #3.In this document, the citations are in the format (surname et al., year), which is more informative, while in the revised manuscript, the citations are superscript numbers, which is more compact.
Reviewer #1 -Role of gender in disease, systematic reviews (Remarks to the Author): In principle the aim of the work is very interesting and covering aspects that have not been investigated in detail as the sex difference in outcomes after the oncological drugs Response: We thank reviewer 1 for the positive assessment and especially for recognizing that our study investigates sex differences in oncology clinical trials in more detail than has been done previously.
But some questions arise.

a)
It was examined "trialtrove" for a long period and many things changed greatly including law and norms and knowledge on the importance of sex and gender.Therefore, in my opinion, the stratification of this interval in subinterval linked to change in norm and law including FDA norm because the main topic is drug effect and would be of interest to know registration and post-registration trials and if this stratification changes the results.

Response:
We agree and stratification by two-year periods was done in Supplementary Figure 1 in our initial submission, but we did not emphasize the findings that the proportion of trials with sex comparisons has declined steadily in the past 15 years or so.The source data for Supplementary Figure S1 have now been added in the new Supplementary Table S4.From Supplementary Figure 1, we see that the proportion of trials with sex comparisons, especially the most important Phase III trials declined since 2002-2003and especially between 2008-2009and 2016-2017.We rewrote and expanded the pertinent text in Results to read as follows: The proportion of trials with sex-specific comparisons decreased over time in trials of all phases, and this finding was consistent when trials were further split to Phase II and Phase III only (Supplementary Figure 1, Methods, Supplementary Table S4).The decline was especially substantial between trials with starting years in 2008-2009 (58/2030 trials had sex comparisons) and trials with starting years in 2016-2017 (20/1985 trials).This difference in proportions is highly significant Fisher's exact test,.We suggest that there are two overlapping reasons for this decline.First, as explained in the Discussion, in the first decade of the 21 st century caution among biostatisticians about subgroup comparisons increased, especially regarding post hoc subgroup comparisons.Second, as we observed in the second paragraph of Results, recent trial papers tend to report separate analysis of males and females as well as other subgroups but tend to avoid direct comparisons between subgroups.For readers who wish to analyze the trends using time intervals different for the two-year intervals we used, we provide more fine-grained data by single year and for each trial phase in Supplementary Table S5.As an influential anecdotal example, we mention a meta-analysis of sex and response to immune checkpoint blockade, all 23 of the underlying trials analyzed males and females separately without a direct comparison (Wallis et al. 2019).Of note, we found the highest fraction of candidate trials with sex comparison in the Phase III set (Supplementary Figure 1), suggesting that as drugs near requests for regulatory approval, doing comparisons by sex increases in importance.
It will be useful to known the phase of clinical studies of pre-registration trials Response: Thanks.The combination of phases and starting two-year period was visualized in Supplementary Figure S1 in our initial submission.The source data for Supplementary Figure S1 have now been added in the new Supplementary Table S4.We recognize that our findings in the previous response are unexpected, and readers may want to reanalyze the data using time intervals other than two-year windows.Therefore, we added source data for each combination of (start year, phase) in the new Supplementary Table S5.Both of these new Tables are crossreferenced in the new text in Results, which is copied in our response to the immediately preceding comment from the reviewer.

Response: We thank the reviewer for the idea of including an analysis by age. Regrettably, what we can do with
Trialtrove data is limited by its nature.We added in Results (page 7) a subsection entitled "Limited information about ages" containing the following comments and analysis: Our primary analyses did not consider patient ages, although there are known interactions between sex and age affecting response to treatment (Austad & Fischer 2016;Hägg S. & Jylhävä 2021;Stader & Marzolini 2022;Tannenbaum et al. 2017;Trenaman et al. 2021;Zhernakova et al. 2022).Trialtrove coarsely annotates the ages eligible for a trial as any subset of three values, Children (ages 1-17), and 71,832/89,221 have such an annotation.Among the oncology trials with an age annotation, the vast majority 59,792/71,832 (83%) have the age annotation "Adults; Older Adults".Among the trials with a sex comparison and an age annotation, the proportion annotated as "Adults; Older Adults" is similar at 294/362 (81%).The proportion of all trials annotated as exclusively "Older Adults" is only 1261/59782 (2.1%), precluding any further analysis of significance into these differences.d) Dose is a key point for drug response for both efficacy and safety profile, the paper does not report any information about this point which is essential.administered to men and women.Obviously this is a fundamental point Response: Thanks.The issue of dosage is a very important limitation.We added to the Discussion (page 14) Another major limitation of our study is lack of access to individual patient-level data.This hampers our ability to assess potential confounders contributing to our findings on sex-specific differences, such as patient age or total dose of drug received.The interaction effect between sex and age on morbidity and mortality is well established, with women living longer and experiencing greater frailty in older age, but we are not able to observe the effect of this interaction with the available data (Austad & Fischer 2016;Hägg & Jylhävä 2021;Knufinke et al. 2023;Zhernakova et al. 2022).Similarly, pharmacology studies have demonstrated that pharmacokinetics and pharmacodynamics are different between males and females (Mauvais-Jarvis et al. 2021;Moyer et al. 2019;Madla et al. 2021;Trenaman et al. 2021;Soldin et al. 2011;Tannenbaum et al. 2017;Stadler & Marzolini 2022;Knufinke et al. 2023), with these factors as contributors for women experiencing more side effects (potentially older women more specifically).Regrettably, only 33 of the 472 trials in Trialtrove that performed sex-specific comparisons describe any data collection related to any of the cytochrome P450 genes involved in drug metabolism, precluding any systematic analysis of such interactions.

Line 49 elimination is included in pharmacokinetics
Response: Thanks for this correction.We changed the sentence to use the terminology of (Mauvais-Jarvis et al. Pharmacol Rev 2021 Apr;73(2):730-762): "For instance, body composition and metabolism differences between the sexes might influence pharmacokinetics and pharmacodynamics drugs." Line 50 please add a more references to highlight that the problem extends to other pathologies and it is not limited to cancer Response: We added citations to (Tannenbaum et al. 2017;Moyer et al. 2019;Trenaman et al. 2021).We changed "including cancer" to 'including diabetes, cardiovascular diseases, and cancer" According to https://www.nature.com/documents/ncomms-submission-guide.pdf,"As a guideline, Articles allow up to 70 references" Our initial submission had 56 references and our revised submission has 71 references thanks to the helpful suggestions of reviewer #1 53-62 It is clear what the authors want to underlie but they should recall that study sex differences is not only a question of recruitment of men and women but there is lot of concerns as illustrated by many authors.

Response plan:
We welcome the opportunity and the reviewer's suggestion to expand this part of the Introduction to list additional concerns and challenges in study sex differences in clinical trials.We replaced the single sentence: "To do useful comparisons by sex, requires sufficient clinical trial arm recruitment of both sexes (Mendis et al. 2021) and the need to evaluate treatments in both sexes has regulatory implications (Gispen-de Wied, C. & de Boer 2018)." With the following paragraph (page 3-4): Designing, analyzing, and evaluating clinical trials to examine and compare treatments in both sexes requires careful planning.It is important to recruit sufficiently many males and females to have some power to detect differences (Mendis et al. 2021) (Mauvais-Jarvis et al. 2021, Stader & Marzolini, 2022;Tannenbaum et al. 2017, Trenaman et al. 2017).Differences in death rates and aging patterns should be considered when comparing survival characteristics of males and females, especially when studying diseases, such as cancer, that predominantly afflict older patients (Dong et al., 2020;Hägg, S. & Jylhävä 2021;Lee et al., 2018;Zhernakova et al. 2022).

. Regulators must be clear and consistent about what sexspecific subgroup analyses and sex comparison analyses are expected in filings reporting trial results (Gispen-de Wied, C. & de Boer 2018). Differences in proper doses between males and females should be planned and should also consider the patient's age and pharmacogenomic markers
The discussion is quite generic and should discuss properly the role of age and of the dose Response plan: We revised the Discussion substantially including a paragraph about age and dose, which we copied above, and the following new paragraph about policy implications (pages 14-15): Our analysis showcases the impact and room for improvement in current policies to identify sex-specific results in clinical trials.A 2014 US Food and Drug Administration (FDA) Action Plan (https://www.fda.gov/media/89307/download)highlighted 27 actions divided into the priorities of "improving completeness/quality of demographic subgroup data collection, reporting, and analysis; identifying barriers to subgroup enrollment in clinical trials and employing strategies to encourage greater participation; and making demographic subgroup data more available and transparent."Additionally, projects awarded by the FDA's Office of Women's Health Research will start to address some of the questions we bring up with this work, but this group has the funding for only a limited number of projects with a duration of 1-2 years per project (https://www.fda.gov/science-research/womens-health-research/list-owhresearch-program-awards-funding-year#2024).Prioritization cannot be at the level of the FDA alone.Incentives for recruiting sufficient patients and performing these comparisons must also be at the level of journals.As of 2016, several top-tier scientific and oncology-specific journals and journal families, including The Lancet family, Journal of the National Cancer Institute, the Cell family, the Nature family, and the Science family, have adopted SAGER guidelines that require reporting of sex / gender of participants and, to some extent, justification for inadequate powering for subgroup analysis (Heidari et al. 2016;Schiebinger et al. 2016).SAGER guidelines or some similar alternative should be the norm across journals.
Reviewer #2 -gender disparities, systematic reviews (Remarks to the Author): This study addresses an important set of questions--whether oncology trials report statistical comparisons between sexes and when they do whether they find important differences in outcomes or toxicity.More, its sample is uniquely large, analyzing 89,221 trials in trialtrove.

Response:
We thank reviewer 2 for the positive assessment and especially for recognizing that the questions in our study about sex differences in clinical trials are important.
However, the paper is a bit hard to follow, for a few reasons, which are addressable.First, it would be helpful to have a formal methods section between the introduction and results sections of the paper.Currently, some methods are briefly described in the introduction and the results section, but the actually methods section is at the end of the paper.

Response:
Thanks.We also share reviewer #2's preference to have the section order as Introduction, Methods, Results, Discussion.However, according to the instructions for Nature Communications articles at https://www.nature.com/ncomms/submit/article, the required order of sections is Introduction, Results, Discussion, Methods.The reviewer perceived correctly that we gave a short overview of the methods in Introduction and Results aiming to guide readers who are methods-oriented.To address the reviewer's concern given these guidelines, we have now explicitly mentioned in the Introduction (page 5) and Results (page 6) that "methodsoriented readers are encouraged to skip from here to read all of the Methods section and then come back to this paragraph." Further there seem to be some new results presented in the discussion section (ie the total number of patients enrolled in the analyzed trials).

Response: Thanks for pointing this out. Considering your comment, we now moved the information about total enrollment to page 6 of Results.
There are some fragments and incomplete sentences in need of revision, for example, see line 87, which states, "Recent large studies that similarly used Trialtrove include." Response: Thanks.We completed the sentence on line 87, now on page 5, to read: Recent large studies that similarly used Trialtrove include a study predicting drug approvals, a study about the use of germline information in clinical trials, and a catalog of immunotherapy trials. 43-45  We identified some other sentence fragments and sentences that were too long and rewrote them into short full sentences.
In terms of the analyses, it could be helpful to add a sensitivity analysis comparing reporting of statistical comparisons between sexes in clinicaltrials.govand or publications with what is in trialtrove.As currently structured, the paper reads more about what is curated by trialtrove than more generally in clinicaltrials.govor the medical literature.If these data exist and trialtrove has not prioritized curating them -that could be an important finding.Without the analysis I suggested I am unsure how representative these findings are of real world practices versus trialtrove curation practices.Overall, an important topic.

Response: Thanks. The reviewer raises two important questions of quality control and alternative sources of information.
First, does ClinicalTrials.govinclude sex comparisons of outcomes?The answer is very rarely for several reasons.We did additional curation analysis and we added the following to Results (page 6): In general, Trialtrove contains substantial information absent from ClinicalTrials.gov.This is especially true for sex comparisons of results, for which ClinicalTrials.gov contains almost no information.We evaluated on September 12, 2023 all 316 trials for which we found at least one sex comparison with a difference (not necessarily statistically significant).Among these 316 trials: 90 are not in ClinicalTrials.govat all because they are in registries outside the USA, 142 are in ClinicalTrials.govwithout any results, 79 mention sex only with respect to enrollment, which is called "Participant Flow", and only 5 mention sex anywhere other than Participant Flow.Only 1 of these (NCT00418886) has separate analysis by sex.Ironically, the clearest indication that ClinicalTrials.govdoes not record sex comparisons is trial NCT01274338 in which the associated paper entitled "Enhanced immune activation within the tumor microenvironment and circulation of female high-risk melanoma patients and improved survival with adjuvant CTLA4 blockade compared to males" (Saad et al. 2022) describes the sex comparison in the title; this paper is listed in the publications associated with trial NCT01274338.However, the Results subsection of the ClinicalTrials.goventry for NCT01274338 has no analysis by sex.We recognize that ClinicalTrials.govhas neither the staff to do the curation nor the enforcement powers to require clinical trialists to deposit their results, so the point of this analysis is to quantify the value added by the Trialtrove curation of thousands of sources.

Second, does Trialtrove capture most of the sex comparisons available in publications? We did such a sensitivity analysis of 75 trials for the original submission and found that Trialtrove curation almost always finds any statistically significant differences in the paper main documents but misses some comparisons that yield non-significant results, especially if they are in the supplementary information. Regrettably, we relegated most of this sensitivity analysis to the Supplementary Information (including what was Supplementary Table 2) in our original submission. We now put more of this information into the main document including converting that Table to what is now Table 1, placed after the References. We substantially enlarged and completely rewrote the main Results text about the sensitivity analysis as follows (pages 7-9).
Our analysis relies on Trialtrove curation, which could miss sex comparisons.Therefore, after doing most of our search curation, we selected 75 trials with large enrollments ≥ 200 including both males and females that appeared to have Trialtrove-curated results but did not have a sex comparison identified by our analysis (see Supplementary Methods subsection entitled "Quality Control of Query and Preliminary Results").It is important to understand that these are among the largest oncology trials done and were chosen because the large sizes imply that the trials are likely to have sufficient statistical power to detect a sex difference in outcomes if one exists.We intentionally did this quality control analysis shortly before finishing our curation so that if we found possible improvements in our methods, we could implement those improvements and we did implement one improvement.
To this end, we searched in detail any papers and abstracts published about those 75 large trials to see if any sex comparisons were missed by Trialtrove curators.Considering that we found fewer not significant comparisons than significant comparisons (Figure 1b), we expected ahead of time that Trialtrove curators may have missed some not significant comparisons.The results of our quality control analysis are in Table 1 (placed after the References).
For only 1 of 75 large trials we checked, there was a statistically significant sex comparison in the main document of the publication that Trialtrove curators missed.We infer that Trialtrove curators found the large majority of statistically significant published sex comparisons.As expected, there was a larger number of trials (8/75) that had an insignificant (7/75) or marginally significant (1/75, whether it is significant depends on not correcting for multiple tests) sex comparison relegated to the supplementary information; hence, Trialtrove curators missed the sex comparisons in those 8/75 publications.For the other 66/75 trials, no sex comparison was published.Importantly, a larger proportion of papers (17/75) had a subgroup analysis by sex in which males and females were analyzed separately but no comparison was done.In the next paragraph and in the Discussion, we hypothesize as to why we think this practice of subgroup analysis of males and females separately with no sex comparison has arisen.
At the suggestion of a reviewer, we added a second, larger assessment of all 147 trials that were eligible for our main analysis and were not found by us to have a sex comparison curated in Trialtrove, appeared to have results, and had enrollments in the slightly smaller range of [175,199].The enrollment criterion was selected to prefer large trials that have power to find sex differences while avoiding any overlap with the first assessment, which required enrollment ≥ 200.Encouragingly, we found only 2/147 trials with a significant SOR or side effect sex comparison that Trialtrove curators missed and 0/147 that our search methods missed (Table 1).The main differences in the outcomes of the two quality control assessments were an increase in the proportion of studies with a paper but no analysis by sex (51% in the first assessment and 65% in the second assessment) and a corresponding decrease (23% in the first assessment to 7% in the second assessment) in the studies with a paper that did a separate assessment of males and females.Possible reasons for this difference include that i) larger trials are more likely to have power to analyze males and females separately and ii) larger trials are more likely to be published in very high impact journals such as New England Journal of Medicine and Journal of Clinical Oncology, which have developed standardized, in-house figure designs for forest plots that are used to illustrate subgroup analyses, such as separate analyses of males and females.Thus, it appears that authors who publish their trials in the top journals look at how subgroup analyses in their journal of choice were done and imitate previous publications that analyzed males and females separately.The trials with enrollment < 200 are naturally less likely to be published in the highest impact journals and the lower impact journals do not necessarily encourage authors to do analyses by sex, either separately or in comparison (see Discussion).

Reviewer #3 -Cancer clinical trials (Remarks to the Author):
This is a retrospective analysis to explore the outcome differences by sex in oncology clinical trials.Generally, this study highlights an important issue that statistical comparisons between sexes in oncology trials are rarely reported, meaningful differences in outcome and toxicity exist.To some extent, this paper is well-written and has relatively high clinical implications.

Response:
We thank reviewer 3 for the positive assessment and especially for recognizing that the issue of sex differences in clinical trials is important.
However, the manuscript can be improved and the following comments need to be adequately addressed: 1.For figure 2-4, the authors indicate the outcome differences of clinical trials through "male", "female", and "same".I cannot think the outcome differences of these trials indicated by the "same" reported the same outcome differences between male and females.In fact, it is that the differences are not statistically significant.So the authors should kindly revise this information.
Response: Thanks.We changed 'same' to "not different" in  The evidence type is Other Numerical Comparison or No Numerical Comparison and the curation described the comparison with an adjective such as "same" or "similar" or "nearly identical".
2. Authors revealed that only 472/89,221 oncology clinical trials 25 (0.5%) had curated posttreatment sex comparisons.Whether this result indicate the sex difference analysis is not mandatory for clinical trial design?If yes, whether the conclusions of this study can be applied for further trials?
Response: Thanks for raising this central and important point.As we explained at the beginning of the Introduction, sex difference analysis is mandated by law in the United States.Sex difference analysis may not be mandated in other countries.One of our key conclusions is the current United States law is not being enforced.Therefore, we added in Results (page 10): It may be argued that the denominator of 89,221 is an overestimate for several reasons.Therefore, we counted the subset of trials that had an enrollment of more than 25, enrolled both males and females, had a known start year of 1993-2022, and had S6.
We moved the previously done analysis over time (Supplementary Figure 1) to follow the above new paragraph.In general, we see no improvement in more recent years.One of our key conclusions is that in the United States the current law and NIH policy are not being enforced.We also noted (page 10) that: To summarize the analysis over time, we do not observe any increase in the proportion of United States -based trials with a sex comparison after any of four key events: enactment in 1993 of the law mentioned above, establishment of ClinicalTrials.gov in 2000; requirement of trial registration in ClinicalTrials.govstarting around 2007, and implementation of the NIH policy on sex as a biological variable around 2016.
3. In Methods, authors should the clarify the update time of ClinicalTrials.gov in Trialtrove database.Meanwhile, please provide detailed information when using Python.

Response: Thanks. We added on page 16:
Trialtrove is updated every weekday and in our experience, new data in ClinicalTrials.govappear in Trialtrove within days or a few weeks.To have a clear and consistent reference point, we had to take a single data freeze (in December 2022) to have a stable set of data to analyze.
We now make available via GitHub two python programs to process Trialtrove data.One program formats the downloaded Excel files into text files that are more easily searched for some purposes.The other program was used to find the initial set of 4,061 candidate trials that may have a sex comparison.This is mentioned in Code Availability on page 19. 4. Authors should provide more in-depth discussion of how this study can impact existing and further clinical trial policies.

Response plan:
We have substantially rewritten the Discussion, including mentioning the issues of age and dose.Regarding policies, we added this paragraph (pages 14-15): Our analysis showcases the impact from and room for improvement in current policies to identify sex-specific results in clinical trials.A 2014 US Food and Drug Administration (FDA) Action Plan (https://www.fda.gov/media/89307/downloadhighlighted 27 actions divided into the priorities of "improving completeness/quality of demographic subgroup data collection, reporting, and analysis; identifying barriers to subgroup enrollment in clinical trials and employing strategies to encourage greater participation; and making demographic subgroup data more available and transparent."Additionally, projects awarded by the FDA's Office of Women's Health Research will start to address some of the questions we bring up with this work, but this group has the funding for only a limited number of projects with a duration of 1-2 years per project (https://www.fda.gov/science-research/womens-healthresearch/list-owh-research-program-awards-funding-year#2024).Prioritization cannot be at the level of the FDA alone.Incentives for recruiting sufficient patients and performing these comparisons must also be at the level of journals.
As of 2016, several top-tier scientific and oncology-specific journals and journal families, including The Lancet family, Journal of the National Cancer Institute, the Cell family, the Nature family, and the Science family, have adopted SAGER guidelines that require reporting of sex / gender of participants and, to some extent, justification for inadequate powering for subgroup analysis.(Heidari et al. 2016;Schiebinger et al. 2016).SAGER guidelines or some similar alternative should be the norm across journals.
the text and the more compact "no diff." in the Figures.In the Supplementary Methods, page 5, we modified and added the following text to clarify: Additionally, when annotating comparisons for evidence type used, the possible options are Multivariate Analysis, Univariate Analysis, Other Numerical Comparison, No Numerical Comparison.Using these categories, a trial has a comparison that is not different ("no diff." in the Figures) if either: at least one data collection location in the United States including Puerto Rico.The last two requirements are because the United States Public Health Service Act sec.492B, 42 U.S.C. sec.289a-2 (quoted at the start of the Introduction) was enacted only in the United States in 1993.The number of trials in the numerator and denominator meeting the four requirements above are 215/17988 (1.2%).The numbers for each start year are shown in Supplementary Table